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(54) Automatic color saturation enhancement 

(57) A method of processing a digital color image 
comprising the steps of: providing a digital color image 
having pixels and a plurality of color channels; process- 
ing the image to provide a channel independent restored 
digital image; determining at least one predictor derived 
from the channel independent restored digital image or 
the digital color image; determining a channel interde- 
pendent color boost relationship from the at least one 
predictors; and applying the color boost relationship on 
a pixel by pixel basis to the respective channels of the 
digital color image to produce a processed digital color 
image. 
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Description 

[0001] The present invention relates to methods of processing digital color images, and more particularly to a method 
of adjusting the color saturation of a digital color image. 

[0002] In photographic printing of film, such as a color photographic negative, it is a well-known practice to 'correct 1 
the color balance by causing the overall color balance of the print to be a shade near gray. This correction strategy is 
based on the assumption that the overall average of the scene integrates to a gray color and is very effective at reducing 
he effects resulting from different scene illuminants that are spectrally different such as tungsten and daylight In a 
like manner image sensing apparatus such as a video camera, average, over a time period, color difference signals 
H-Y and B-Y, to a zero value. This is equivalent to integrating to gray. 
[0003] These methods work well for the majority of scene and illuminant combinations. However, when the scene 
subject matter is highly colored, particularly with a single dominant color, the integrate to gray strategy fails as this 
dominant scene color is mistaken for an illuminant bias. This failure, known as subject failure, produces unpleasant 
colored casts in the color complimentary to the dominant scene color. There are various strategies for minimizing these 
failures. These strategies are typically based on reducing the amount of correction based on a population of images 
and/or on the information in neighboring frames. Other color problems result from fading of the dyes in a photographic 
image, printing and processing errors, film keeping problems, and in the case of a color digital image that is captured 
directly with an electronic camera, a misadjusted black point in the camera. 

[0004] With a digital image, obtained either directly from an electronic camera, or indirectly by scanning a photo- 
graph.cprintorf.lm,itispossibletomanuallyadjustthecolorbalancebyusinganyofthewell known digital photographic 
manipulation tools such as Adobe Photoshop ®. However, manual adjustment is not practical for automated printing 
of digital images. A digital image provides much information that can be used for calculating color adjustments and 
several methods have been proposed for performing these adjustments automatically. Some of these methods such 
as that taught in US Patent 5,555,022, issued Sept. 1 0, 1 996 to Haruki et al. divide the scene information into a plurality 
of regions representing different locations within a scene. Means to select and weight the correction of these blocks 
are then employed to provide automatic white balancing and to restrict the degree to which color correction gain is 
^ ! f J° u 8ZaWa ' US PatSnt 4 ' 984 ' 071 teaches a melhod ^ adjusting a gradation curve based on identifying shadow 
and highlight points by utilizing histogram data. Morikawa, US Patent 5,062,058 describes a system that uses a cu- 
mulative histogram to designate highlight and shadow points. Other histogram-based methods are taught by Lin US 
Patent 5,81 2,286 and Edgar, US Patent 5,265,200. Edgar further describes a method performing a second order best 
frt regression of the histogram data and includes methods to eliminate some histogram values from consideration in 
the regression. 

[0005] Another approach that combines color correction with tone scale corrections is based on random sampling 
within a digitized image and subsequently modifying the resulting histogram of these samples. US Patent 4 677 465 
issued June 30, 1 987 to Alkofer and US Patent 4,729,01 6, issued March 1 , 1 988 to Alkofer disclose relatively complex 
methods that utilize these samples in a plurality of segmented contrast intervals through noimalization techniques and 
with comparison to image population data. 

[0006] An improved method for simultaneously adjusting color scale and color balance was disclosed in US Patent 
Application S.N. 09/650,422, filed Aug. 29, 2000, that preferably calculates gain and offsetfactors by regressing masked 
red, green, and blue digital image data to a masked reference digital image. 

[0007] While the above described methods can correct for tone scale, they do not provide method or means for 
enhancing color degradation resulting from the loss of color purity or color saturation. This type of color degradation 
can be restored if channel interdependent image processing, such as a 3x3 color matrix, is applied 
[0008] US Patent 5,796,874 by Woolfe et al. describes an approach that is specifically designed to correct for dye- 
fade in Ektacolor paper includes generating a 3x3 restoration matrix that can compensate for this loss of purity or color 
saturation. The disclosed restoration matrix is a matrix of coefficients of second order polynomials in time with factors 
that are optimized for specific paper, are applied in logarithmic space, and which compensate for the image-wise liqht 
filtration from overlying layers in the film. The compensation matrix is followed by tone-scale adjustments This method 
assumes that additional information, time and type of photographic material, are known. This information is oftentimes 
not available. In addition, parameters other than time, such as temperature and light exposure, can impact stability 
and long term purity of dye images. 

[0009] Another method to generate a color restoration matrix is taught by Edgar, US Patent 5,673 336 wherein 
determining the correlation of the noise in the digital image is used to generate and apply a decorrelating matrix to 
restore the image. This method has the advantage that it does not require any additional information on the pedigree 
of the image; however, it assumes that there is a detectable noise signal, that this noise signal is not contaminated 
with image signal, and that in the original, non-degraded image, the image noise was fully decorrelated and was of 
near equal power levels amongst the channels. It is difficult, or in many cases impossible, to guarantee that these 
assumptions are met. 
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[0010] It has been further observed that when digital images of dye faded images have been processed through 
many of the above methods that provide both gain and offset tone scale corrections to each channel that a uniform 
color boost among the color channels, color boost, in which a color restoration matrix with all diagonal terms equal 
and with all off-diagonal terms equal, is effective at restoring color saturation. One embodiment of the method taught 

5 by Katsumain US Patent 5,398,1 23 uses such a uniform boost to increase color saturation. Katsuma teaches a method 
and apparatus to increase color saturation wherein at each pixel, the difference between the maximum value amongst 
a plurality of color components at that pixel and the minimum value amongst a plurality of color components at each 
pixel is calculated. These calculated differences for each pixel are averaged for the entire image, and if this average 
exceeds a threshold, a correction factor, based on the dynamic range of the image and this average, is applied to the 

10 image as a uniform saturation boost. This saturation boost is applied as a global change, that is, every pixel is subjected 
to the same image processing. He notes that this operation can result in overflow problems and solves these problems 
by mapping overflow in any of the color components at any pixel to the maximum or minimum value supported by the 
image processing means, e.g. 255 and 0 respectively for an 8 bit per color component imaging system. Furthermore, 
his method requires that an operator select the above-mentioned dynamic range at the initial step in the enhancement 

15 process. 

[001 1] In a similar approach to enhance color saturation, Shu in US Patent 5,51 7,335 teaches a method wherein at 
each pixel, the maximum value and the minimum value amongst a plurality of color components are calculated. His 
method differs from that disclosed by Katsuma in that Shu calculates a modification to each pixel based on only the 
color component values at that pixel. The difference between the maximum and minimum value at a pixel and the 

20 average of the code values for all of the color component values at that pixel are inputted into their respective look up 
tables and the product of the output from these look up tables forms a delta value. He notes that this delta value is 
multiplied by a predetermined constant that allows the effect of the enhancement to be varied to suit a particular image. 
The saturation of the image is enhanced by subtracting this delta from the color component with the minimum value 
at that pixel and adding this delta to the color component with the maximum value, thus preserving the average amongst 

25 the color components. Shu notes that overflow problems can occur and solves these problems by designing the look 
up tables so as to minimize these problems. 

[0012] Both the method of Katsuma and the method of Shu base their color saturation increase, i.e. color. restoration 
on values, at any pixel, from only two color channels. This can result in biases if the third color, in the case of a three 
color red, green, and blue digital image, is very close in value to either of the two values that are used in determining 
30 the color restoration, then the overflow problem noted above is exacerbated. In contrast to the uniform boost methods, 
the method of Shu, if the value from the third, unused color value is close to either of the used maximum or minimum 
values, then hue shift will occur as only these two colors are modified, when all three must be modified in order to 
maintain color hue. 

[0013] To date, none of these techniques for modifying color saturation have proven entirely satisfactory in addressing 
35 the problem of automatically restoring color purity in a digital image, particularly where the digital image has been 
derived by scanning an image that has experienced severe dye fade. 

[0014] There is a need, therefore, for an improved digital image processing method for automatically adjusting the 
color balance of a digital image. 

[0015] The need is met according to the present invention by providing a method of processing a digital color image 
40 having pixels and a plurality of color channels, comprising the steps of: processing the image to provide a channel 

independent restored digital image; determining at least one predictor derived from the channel independent restored 

digital image or the digital color image; determining a channel interdependent color boost relationship from the at least 

one predictors; and applying the color boost relationship on a pixel by pixel basis to the respective channels of the 

digital color image to produce a processed digital color image. 
45 [0016] The present invention has the advantages that it does not require knowledge of the pedigree of the image,. 

that it is computationally efficient, and that it avoids the problems of generating pixel value overflow, selecting dynamic 

range, selecting image type, and/or creating color hue shifts. 

Fig. I is a schematic block diagram illustrating an image processing system suitable for performing the image 
50 processing method of the present invention; 

Fig. 2 is a schematic diagram illustrating the digital image processing method according to the present invention; 

[0017] Referring to Fig. 1 , an image processing system, generally designated 1 0 suitable for performing the image 
processing method of the present invention includes for example an input device 12 for producing a digital image, such 
55 as a scanner for scanning photographic film or prints, or a digital camera. Alternatively, digital images may be sourced 
from a digital data storage device and accessed via the World Wide Web. The input device 1 2 produces a digital image 
signal having a plurality of color channels (e.g. red, green and blue) that is supplied to a digital image processor 16. 
The digital image processor is a programmed computer and can be for example a general purpose digital computer 
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such as a PC, or minicomputer, or a computer that is specially designed for processing digital images. The digital imaqe 
processor 1 6 is provided with operator interface devices such as a keyboard 1 8 and a mouse 20. The system 1 0 may 
include display devices, connected to the digital image processor 16, such as a color monitor 22 and a output device 
24, such as a color hardcopy printer (e.g. an ink jet or thermal printer) for producing a color hard copy print 26 of the 
processed digital image. The system may also include a digital image store 28, such as an optical disk storage unit 
for storing the processed images, and a communications connection 30, such as a modem, for communicating the 
images to a remote location. 

l °W , R i eferring to Fig - 2 ' the ima 9 e Processing method of the present invention includes the steps of providing so 
a digital color .mage having pixels and a plurality of color channels and processing 31 the digital color image to provide 
a channel independent restored digital image. The method described in U.S. Patent application S.N. 09/650 422 is 
such a channel independent method. Other examples of channel independent methods include those taught in US 
Patent 5,555,022, US Patent 4,984,071 , US Patent 5,062,058, and US Patent 5 812 286 
[0019] Next predictors are determined 32 from the digital color image and / or the channel independent restored 
digital image. Predictors are chosen based or their ability to identify digital images where saturation boost is desired 
15 Preferred predictors include. 

[0020] Preferred predictors based on the channel independent restored image include predictors that are global in 
hat information from all of the pixels are used, and predictors that result from using data from a pixel that represents 
the maximum or minimum of a certain determination. 

[0021] An example global predictor is the ratio of the standard deviation of luminance representation of the digital 
color image divided by the mean length of the chrominance vector. The dynamic range of the image is noimalized by 
using the standard deviation of this luminance representation, W, which is defined as: 
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W= x-R + y-G + (1 -x-y)B 
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where R, G, and B are, respectively, the red, green, and blue digital color image and x and y are weighting factors 
each greater than zero and their sum less than one. When xand yboth equal 1/3, then the luminance representation 
is an equal we.ght.ng of red, green, and blue. The standard deviation, o w of the luminance representation is calculated 
by well known statistical methods and is: 



a _ k(w- 



W) 2 



where n is the number of elements, e.g. pixels, used to calculate the mean of the luminance representation W and 
the summation is over all pixels in the image. 

[0022] There are various ways to represent a chrominance vector which is defined as an absolute value deviation 
from neutral where all three code values are equal. An example representation for a chrominance vector, C, is: 



C= J(R-W) 2 + (G-W) 2 + (B-W) 2 
and the average chrominance vector, C defined as: 



(3) 



C ~ T (4) 



The predictor, P v will have a lower value when the standard deviation of the luminance representation has low value 
and the average chrominance vector has high value: 



P 1 " *1 ~C (5) 



55 where /C, is a weighting factor for predictor . 

[0023] Another example of a global predictor, P 2 , is defined as: 
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where K 2 is a weighting factor for predictor P 2 and the min( ) operation selects the minimum standard deviation among 
three color minus luminance signals, R-W, G-W, and B-W. 

[0024] An example predictor resulting from a maximum or minimum determination based on analysis of all pixel 
values is P 3 with weighting factor K 3 : 



K 3 

P * ~ max(|fl-G|,|G-fl| t |S-fl|) 



(7) 
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and the max( ) operation selects the maximum absolute value color difference among the red, green, and blue color 
values at each pixel and then maximum of these pixel level maximums among all pixels in the image. 
[0025] Another example predictor resulting from a maximum or minimum determination is based on a determination 
that limits, or alternatively eliminates overflow or underflow by utilizing the relationship among all three color values at 
each pixel. Examination of the uniform color boost equation: 



(8) 



where sis the uniform boost that is applied to the input R, G, and 8 values resulting in the output Ft' , G\ and & values. 
To avoid overflow, all of the 3 values /?', G\ and B must not overflow either above a maximum value l/ max or below a 
minimum value V min . There are 6 possibilities for the relationship among the red, green, and blue values: 

R> G,B (9a) 

G>R,B (9b) 

B>R,G (9c) 

R<G,B (9d) 



G<R,B 
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(9e) 
(9f) 



B<R,G 

Examining 9a and matrix equation 8, maximum value overflow is controlled if the following inequality is satisfied: 

1 2R-G-B 

s > V^R (10a > 

Similar inequalities that result from 9b and 9c are: 

5 > WS5fG (10b) 
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1 2 B-G-/ ? 

s > Vmax-B ( 10c ) 



Equations 10a, 10b, and 10c are evaluated as equalities at every pixel and the maximum value of V. from these 3 
equalities is chosen to limit the value of s that causes maximum value overflow. 

In a similar manner, by examining 9d, minimum value overflow is controlled if the following inequality is satisfied: 



Similar inequalities that result from 9e and 9f are: 



1 ^ 2R-B -G 

s > Vrmn-R ( 10d > 



1 2G-R-B 

s > l/min-G ( 10e ) 



1 2B-R-G 

s"* l/min-B ( 10f ) 



Equations 10d, 10e, and 1 0f are evaluated as equalities at every pixel and the maximum value of V s from these 3 
equalities is chosen to limit the value of s that causes minimum value overflow. 

[0026] As maximum overflow resulting from a saturation boost is oftentimes more visually objectionable than a min- 
imum overflow, the maximum value of V s based on the equality form of equations 10a, 10b and 10c forms a useful 
predictor. Also, as some, but controlled, overflow may be desirable and given that this predictor can be used in com- 
bination with other predictors, such as those previously described, the values for l/ mjn and V max are typically set to be 
a few code values beyond the maximum and minimum code value limits. This is also computationally useful as this 
avoids a division by zero should the value of R, G, or B equal the minimum or maximum code value limit. 
[0027] An example predictor P 4 , based on limiting the maximum value overflow is: 



P 



and K 4 is a weighting factor for predictor P 4 and (V s ) max is the maximum value, base on evaluating 1 0a, 1 0b and 1 0c 
as equalities at every pixel in the image. 

[0028] Yet another predictor P 5 utilizes a spatially filtered high-pass frequency representation of the chrominance 
signals: 

P 5 = ^max[(f?-HO hjghpass , (G-MO hlghpassi (B-l^) highpass ]} (12) 

where F{x,y,z] is a function such as forming the average or calculating the maximum value of three arguments x, y, z, 
( R ~ W)high P ass> (G - W)high P ass> and (B- W) are spatially filtered, preferably with zero response at zero spatial frequency 
chrominance signals and K s is a normalizing constant. As was the case with predictor P A the performance of other 
predictors may be enhanced by normalizing by the standard deviation a^of the luminance representation. 
[0029] As this is a predictor and not an operator on the image, a spatial highpass filter as simple as a one-dimensional 
differentiator can be used. For similar reasons, all of the above predictors can be determined from low spatial resolution 
representation of the digital image. Low spatial resolution representation have the advantage of requiring fewer com- 
putations to calculate a predictor and have the additional benefit of reducing any contribution of noise in calculatina 
the predictors. * 

[0030] The above described predictors can be used individually or preferably in combination to determine a final s 
value to applied as a uniform color boost. Those skilled in the art will appreciate that methods that combine (including 
linear and non-linear models,) predictor values in order to better predict aim s values can be used. An example com- 
bination was determined by calculating the above described predictors for images where the uniform color boost s 
was determined by manually adjusting s for each image until image quality was optimum for that image. Details for 
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this example combination, as applied to a digital imaging system wherein values range from 0 to 255 are: 



S =-.0005 + .0173 P t + .0313-.P, -0116-.P 3 
- .0037 • /» + .0002 • P 4 2 - .0005 •/> •/> /> 



(13) 
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where S is the boost factor to be applied to generate the color boost relationship using an equation like equation (8): 
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[0031] As equation (1 3) contains negative coefficients and product terms, it is desirable to limit the output such that 
S is greater than or equal to zero and S less than a high limit such as S less than 1 .0. 
[0032] The following values were used in calculating the predictors used in equation (13): 

= 1 .0 
K 2 = 100.0 
K 3 = 100.0 
K 4 = 1 .0 

[0033] The color boost relationship is applied 36 on a pixel by pixel basis to the channel independent restored image 
30 to produce the color restored processed digital color image R'G'B' 38. 

[0034] In addition to the above detailed examples of masks other masks based on spatial frequency content or other 
digital image processing of pixel data, image statistics and the like can be used to further improve the performance of 
the method. Furthermore, additional predictors may be based on parameters derived from the initial channel independ- 
ent restoration step and/or the original, non channel independent restored image. 

[0035] The present invention is useful for example for correcting the color reproduction of images that have experi- 
enced dye fading, images that were exposed under water or with tungsten or fluorescent illumination using daylight 
balanced film, or images derived from photographic prints that were incorrectly printed. 

[0036] The present invention is preferably practiced in an image processing system including a source of digital 
images, such as a scanner, a computer programmed to process digital images, and an output device such as a thermal 
or inkjet printer. The method of the present invention may be sold as a computer program product including a computer 
readable storage medium bearing computer code for implementing the steps of the invention. Computer readable 
storage medium may include, for example: magnetic storage media such as a magnetic disc (e.g. a floppy disc) or 
magnetic tape; optical storage media such as optical disc or optical tape; barcode; solid state electronic storage devices 
such as random access memory (RAM) or read only memory (ROM); or any other physical device or medium employed 
to store a computer program. 



Claims 



1 . A method of processing a digital color image comprising the steps of: 

50 

providing a digital color image having pixels and a plurality of color channels; 
processing the image to provide a channel independent restored digital image; 

determining at least one predictor derived from the channel independent restored digital image or the digital 
color image; 

55 determining a channel interdependent color boost relationship from the at least one predictors; and 

applying the color boost relationship on a pixel by pixel basis to the respective channels of the digital color 
image to produce a processed digital color image. 
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2. The method of claim 1 wherein said color channels of said provided digital color image are red, green and blue. 

3. The method of claim 1 wherein said at least one predictor P, is 



P« =K 



a 



w 



C 



where w is a luminance representation, a^is a standard deviation of a luminance representation, 
10 C is a chrominance vector, and 

K 1 is a very weighting factor for P 1 . 

4. The method of claim 3 wherein W is defined as 



W= xR+y-G+(1-x-y)B 

where R, G and B are, respectively the red, green and blue digital color channels; and 
where x and y are weighting factors, each greater than zero and x + y is less than 1 . 

5. The method of claim 4 wherein x and y both equal 1/3 and W is consequently an equal weighting of R, G, B. 

6. The method of claim 3 where 



(n-1) 

where n is the number of pixels used to calculate the mean co } of the luminance representation W and the 
30 summation is over all pixels in the color image. 

7. The method of claim 3 where 

35 p K 3 

3 max(|tf-G|,|G-B|,|B-fl|) 

8. The method of claim 1 wherein said at least one predictor P 2 is 

K 0 



P - 2 



W-min(o (/? _ w)>°(G- mo^s- iv>) 

where K 2 is a weighting factor for P 2 and the min ( ) operation selected the minimum standard deviation a 
45 among three color minus luminance signals, R-W, G-W, B-W. 

9. The method of claim 1 wherein said at least one predictor P 3 is 

so p K3 

3 max(|fl-q,|G-B|,|B-fl|) 

where K 3 is a weighting factor and the max ( ) operation selects the maximum absolute value color difference 
among the red, green and blue color values at each pixel, and then maximum of these pixel level maximums among 
55 all pixels of the color image. 

10. The method of claim 1 wherein the channel interdependent color boost relationship is 
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where s is the uniform boost applied to the input R, G, B values resulting in the output R\ G\ B' values. 

11. The method of claim 10 wherein the predictor limits the values R\ G\ B ( to be no greater than a maximum value 
v max or no ,ess tnan a nninimum value V min 

12. The method of claim 11 wherein R > G, B and V max is not exceeded when 

1 > 2 R-G-B 
s Vmax-R' 

13. The method of claim 11 wherein G > R, B and V max is not exceeded when 

1 > 2. G -B B 
s l/max- G ' 

14. The method of claim 11 wherein B > R, G and V max is not exceeded when 

1 2B-G-R 
s Vmax-B' 

15. The method of claim 11 wherein R < G, B and V mjn is not exceeded when 

1 > 2 R-B-G 
s l/min-/? ' 

16. The method of claim 11 wherein G < R, B and V mln is not exceeded when 

1 2 G R-B 
s Vmin-G ' 

17. The method of claim 11 wherein B < R, G and V mln is not exceeded when 

1 > 2 B-R-G 
s Vm\n-B 
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